The present invention relates in general to the field of software product testing, and in particular to a method for automatic testing a software product comprising a plurality of software packages, and a corresponding test environment for automatic testing a software product comprising a plurality of software packages. Still more particularly, the present invention relates to a data processing program and a computer program product for automatic testing a software product comprising a plurality of software packages.
Software products consist generally of several software packages. Some of those packages are used for core functionality of this software product and some of them are optional. Each installation of a new package (mandatory or not) can possibly have influence on already installed packages, for example, through creation of links or through creating of new files or directories in already existing directories, etc. To be sure that a software product is working properly it is important to follow a specific procedure during installation.
To protect the users against wrong installation, the software vendors furnish each package of a software product with installation prerequisites against the other packages inside this software product. This happens during the packaging process as a part of software development. All files needed for a program are combined in such a package. Among other things—like version—the builder specifies also the prerequisites of this software package.
Before the product gets to a customer, it is necessary to test the outcome of the build and packaging process. It is a four-eye principle. A properly developed software product should be able to get installed correctly and should be fully operational afterwards. If there is the need to uninstall a product, this should be possible in a proper way. This should be tested during the installation verification test (IVT). During this installation verification test (IVT), also the dependencies of the product should be verified and tested.
During the installation verification test (IVT) it is important to install or uninstall the software packages of any software product in a certain order, as recommended in the manual. If the dependencies of the packages have been set correctly, the software installation process should be canceled with a helpful error, when a customer is trying to install the packages in the wrong order. Same is true for the dependencies of the packages during uninstallation. The uninstallation routine should also quit with a helpful error, in case of an incorrect order.
There are usually six to eight software packages in one software product. It is not desirable to use a method that calculates all theoretically possible combinations of the package. For eight software packages, for example, there are already 40320 possible combinations, only for the installation on one platform. This number of combinations will rise also for the removal on one platform. Test should try a number of possible combinations of installing and uninstalling. So with every change in the packaging, test needs to retry and change also the combinations of installing and uninstalling tests. Changes of the packaging can occur during every release or program temporary fix (PTF) several times. This leads to massive testing efforts during the installation verification test (IVT).